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The significance of this essay for English educators and research specialists lies 
in its analysis of the demands of and criteria for valid research, frequently ignored 
by researchers in English today. Using as a case in point Roger K. Applebee’s article, 
"National Study of High School English Programs; A Record of English teaching Today’ 
(•English Journal," March 1966), the author suggests that false conceptions and 
stereotypes are perpetuated by the following questionable research procedures; (1) 
using a tone of rhetorical appeal in presentation. (2) selecting a population too small 
to qualify as good random sampling, (3) including only one representative point of 
view in English education, (4) not clearly defining vocabulary terms which have multiple 
meanings nor indicating measurement instruments used in the study, and (5) making 
misleading inferences and conclusions based on insufficient data. The essay then calls 
upon English teachers who are inters ted in undertaking research projects or 
experiments in their classrooms to follow established scientific procedures so that 
their findings can be trusted and accepted. (JB) 
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ASSESSING A NATIONAL HIGH SCHOOL ENGLISH STUDY 

C. James Trotman 



A stereotype of people in the field of English is our inept use 
of procedures for collecting and presenting experimental data. 
The article in the March, 1966, English J oumal, “National Study 
of High School English Programs: A Record of English Teach- 
ing Today,” is no exception. This study is not nationally repre- 
sentative of high school English teaching in today s high. schools. 
There is no question here about the sincerity of those involved 
in the study; but the study’s results lead those who have no 
statistical background to false conceptions, and those who do to 
acknowledge the stereotype. 

The first questionable factor in the article does not involve 
data per se. It centers on the presentation, of the material. At 
best, the presentation is hackneyed rhetoric. 

If I were not in the midst of an age of technology, I should 
now invoke some fiery muse to ascend the brightest heaven of 
invention, to help me turn the accomplishment of these years 
into this little time. Instead, I must simply appeal to your 
imagination, so that when you r-nd numbers or proportions or 
other mathematical abstractions you will cee thousands of 
classrooms, thirty thousands of students, and parades of teach- 
ers in cities, towns and villages all across the country. 

(p. 273). l 

The tone of this reflects the sincerity of the author, but iti also 
reinforces the stereotype of the humanist’s attitude toward re- 
search (i.e. appeal to your imagination, etc.) In short, the pas- 
sage is significant to those who wish that the discipline would 
cease making excuses for itself in terms of research procedures 
and platitudes generally. Furthermore, the passage epitomizes 
the quality of research in the experiment and brings out the sec- 
ond questionable factor for consideration. . 

This factor is the number of schools involved. The number 



i Roger K. Applebee,, •'National Study of High School g n «l (, \K r Xutiont Sd°p.M 
English Teaching Today," English Journal, SB: (March, All quotations »nu v«m 
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of schools (168 out of a possible 30,000) is too few to judge 
high school English teaching; and the schools were selected , 
hence no random procedure was employed. In fact, the article 
reports that the schools were selected on the basis of “. . . their 
state or national reputation . . (p. 273.), with the author 

further stating that the schools not only represent “. . . Excel- 
lence, but extreme variation . . .” (p. 273.) The question is 
rhetorical : How can implications be drawn on the status of high 
school English teaching from a bias sample of so few schools? 
It cannot be done, but it is done here. 

While it is possible to have a small number in an experi- 
mental design, the number’s validity is dependent, in part, on the 
randomness of the sample. One alternative for the researchers is 
to have established procedures for collecting data on kinds of 
schools (academic, vocational or comprehensive, etc.) schools’ 
populations (sociological and economic status, for example) and 
the schools’ settings (rural or urban). The difficulty of this task 
is apparent, but its value to this report’s objectives is not a 
moot point. 

The third questionable factor is extrapolation. From this sam- 
ple, the author maintains that the practices and programs ob- 
served reflect “. . . the diversity, the strengths and the weak- 
nesses of English programs everywhere . . .” (p. 273). The 
point here is that the bias nature of the schools involved does 
not justify this statement. However, the author occasionally in- 
jects sentences to imply randomness or, at best, some feeling of 
objectivity. The following is representative : “Often as many as 
20 classes were observed in a single school” (p. 274). 

Because some schools were visited twice, once by a project 
member and another time by someone outside the project, this 
gave the schools a composite picture “. . . so that we might 
delineate the character and the status of English programs 
today” (p. 274). With this information about the schools, it is 
unjustifiable to make this assumption. This is another example 
of extrapolation. The report is full of similar misleading ele- 
ments. 

The fourth questionable! element in this article is the people 
employed in the program. Although they represent interests in 
the field of education and/or English education, their repre- 
sentation is that of groups in the field who are acknowledged 
conservatives and moderates: the National Council of Teachers 
of English, the National Association of Secondary School Prin- 
cipals and the Commission on English, to name a few. The point, 
however, is that a national study, with its implications for many, 
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would have more validity and appeal if the participants repre 
sented various viewpoints. 

The fifth questionable factor involves methods for 
data. According to the report, observers were not only to w 
descriptions of classes they visited, but also to indicate the P 
portion of time used for facets of English in the classroom, 
the report did not indicate the instruments useu by observers for- 
describing classroom activities, and the proportion of time spent 
on these activities. Furthermore, the imphcation in the stud* is 
that English is being defined as language, literature and com- 
position. While this definition is not unacceptable,^ is not ad- 
vocated universally, witnessed by the definition of Enghs y 
the NOTE and the Commission on English. Thepomt hereis 
that this type of precision is requisite for the study s objectives 
and appropriate for research procedures. 

Aside from the stated time spent on language, literature and 
composition, which allegedly accounted for 82% oi classroom 
time, whatever else went on in classrooms was dismissed to t 
taxonomy of “Other.” Here the point is thar ofcr “ ' m ' 
portant in this report as language, literature and composition, a 
trivium which demonstrates the conservative nature of the pa 
ticipants. Moreover, the report failed to account for certain per- 
centages of time spent on “other” things m the English c s - 
room. And some of the data are set up to look as if the total 
would be 100%, when they do not add up to that figure. 

The alternatives to the described methods of collecting data 

are numerous. Few would argue with the fact that English teach- 
ers are orientated to language, literature and composition, 
had the observers established a list, for example, of things 
look for in addition to the activities mentioned, the report would 
have been more thorough. Furthermore since teachers were m- 
formed ahead of time about observations, the ^hdity of observ- 
ing a routine day is voided. To minimize the Hawthorne Effect 
(or special preparation), teachers could receive notice of ap- 
proximate dates on v/hich they might be observed; a scal ® 
be developed for recording teachers profiles (age, sex, academic 
background and travel experience), kinds of classroom perform- 
ances (techniques) and classroom climates (liberal, authoritar- 
ian, etc.) These suggestions do not exhaust the possibiuties foi 
enhancing the quality of this report, but they do suggest tech- 
niques and procedures for increasing the report s quality, which 
is in no way unrelated to the next factor. 

This factor is definition. Relative to teaching methods, terms 
such aaiWiscussion” and “student preparation” were used which 
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evoke numerous images of classroom interaction. The point is 
that terms with multiple meanings should fee clearly defined. 

The seventh questionable factor is the conclusions made on 
teacher preparation. First the reporting instrument was not 
identified: Was the data obtained by questionnaire, interviews, 
etc. ? What were the criteria used to determine whether or not a 
teacher was an English major? What is an English major? What 
constitutes an English minor? What is the preparation of those 
who fall into the category of “Others?” In the same context, face 
validity (or saying what is possibly true, but without qualifica- 
tion) is involved in maintaining that 50% of the teachers had 
Master's degrees. These selected schools usually attract the more 
qualified because these schools generally pay well, and have pre- 
dominantly middio-class student bodies. To say that teacher 
preparation in these schools is adequate is to understate this 
situation, but it overestimates the preparation of English teach- 
ers across the country. 

The eighth questionable factor involves the teachers. The 
study reported the amount of time teachers spent on marking 
papers and preparing class lessons. The question is what instru- 
ment was used to obtain data for the figures presented on paper- 
correcting and class preparation. 

The ’linth questionable factor is the figures concerning stu- 
dent reading. According to this report, student" are voracious 
readers ,* furthermore, these students were called representative. 
The credibility of this is dubious in light of the whole report. 
Moreover, the libraries considered in this report again pose the 
question of how representative they are of high school libraries 
across the country. 

In the final analysis, the procedures used in the study are 
unsophisticated and awkward. The criticisms outlined here are 
an indication of the study’s deficiencies and it is hoped that there 
are at least two results from ths assessment : First, that a more 
accurate and sophisticated study is made of English programs 
throughout the country ; secondly, that the stereotype of research 
and researchers in the field of English is mitigated. 
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